Markov model

Page numbers are for draft copy at present; they will be replaced with correct numbers when final book is formatted. Chapter numbers are correct and will not change now.

A Markov model is a stochastic process where the probability of each step in the sequence is dependent only on the previous step, or in the case of higher-order Markov models the last n-steps. Markov models are frequently shown as matrices where each entry in the matrix M_ik represents the conditional probability that if the current state is i, then the next state will be k.

Defined on page 323

Used on Chap. 14: pages 323, 324, 325, 326; page xvi